ON FILTERING OF MARKOV CHAINS IN STRONG NOISE 
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Abstract. The filtering problem for finite state Markov chains is revisited in 
the low signal-to-noise regime. We give a description of conditional measure con- 
centration around the invariant distribution of the signal and derive asymptotic 
expressions for the performance indices of the MMSE and MAP filtering estimates. 



1. Introduction 

Consider the discrete time signal/observation pair (X,Y) = (X n ,Y n ) ne z + , where 
the signal X is a finite state Markov chain with values in a real alphabet S = 
{ai, ...,ad}, transition probabilities Xij = ~P(X n = aj\X n _i = m) and initial distri- 
bution v. The observation sequence Y is generated by 

d 

Yn = Y. 1 ix n =a l }Ui), n>\ (1.1) 
1=1 

where £ is a sequence of i.i.d. random vectors, independent of X. Without loss of 
generality the probability laws of the entries of £i can be assumed to have densities 
gi(u), % = 1, d, u G R with respect to a u-finite measure ip{du) on R (typically the 
Lebesgue measure or purely atomic measure). Hereafter all the random variables 
are assumed to be supported on a complete probability space (Q, J^", P). 

This setting is often referred as Hidden Markov Model and is frequently encoun- 
tered in information sciences (see e.g. the recent survey p])- An important statis- 
tical problem related to HMM is filtering, i.e. estimation of the signal X n , given 
the observation trajectory Y up to time n. The main building blocks of this esti- 
mation problem are the conditional probabilities 7r n (i) = P(X n = ai\^^), where 
= a{Y m ^m < n} is the c-algebra of events generated by the observations. 
In particular the minimum mean square error (MMSE) and maximum a posterior 
probability (MAP) estimates of X n are given by 

d 

£r e = X>^n(i) and X%*p = argmax7r n (i). (1.2) 

The vector ir n satisfies the recursive Bayes formula, called the filtering equation, 

GjY^A*^ 

™n = \ n(v ,^ 1» ^0 = V, (1.3) 

|G(y n )A*7r n _i| 

where A* is the transposed matrix of transition probabilities Aj,-, G(y), y £ R is the 
scalar matrix with entries gi{y) and \x\ stands for the ^i-norm, i.e. \x\ = Yli=i \ x i\- 
As usual we identify the probability measures and functions on S with vectors from 



1991 Mathematics Subject Classification. 93E11, 62M05, 62M02. 
Key words and phrases, nonlinear filtering, markov chains. 
Research supported by a grant of the Israeli Science Foundation. 

1 



2 



P.CHIGANSKY 



the simplex S d 1 = {x € R d : Xi > 0, X^i=i x i = 1} anc ^ ^ respectively and use the 
notation rj(f) = J2t=i f( a i)w = f*V for / : S i-> R and 77 G S* -1 . 

While the recursion ()1.3|) provides an efficient way to calculate the estimates in 
(|1.2[) . no closed form formulae are known for the corresponding performance indices: 
the minimal mean square error 

£ n = min E (X n - 9) 2 = E (X n - X™ e ) 2 = EX* - E (vr n (a)) 2 

and the minimum a posterior error probability 

"Pn = min P(X n ^ 9) = 1 — Emax7r n (i). 

6»eL°°(n,,j?i',p) a,es 

and hence approximations of these quantities are of significant interest. 

It is not hard to see that the random sequence ir n is a Markov process with values 
in Under mild assumptions it is also a Feller process and hence it has at least 

one invariant measure M^^drj) (on the Borel field of The uniqueness of this 

measure is not at all obvious and in fact may fail if no restrictions are imposed on 
the noise densities, even when the signal X itself is ergodic (see a discussion in 
Recall that a Markov chain on § is ergodic, if the limit probabilities /Uj := P(X n = 
ai), i = 1, d exist, are unique and positive. The sufficient and necessary condition 
for ergodicity is that the matrix A q has positive entries for some integer q > 1 and 
then \i is the unique solution of A* \i = \i in S^"" 1 . The invariant measure A4 n of ir n 
is unique, i.e. independent of if X is ergodic and the noise densities are bounded 
and have the same support (see [1]). In this case the limits 

£ := lim £ n and V := lim V n 

n—>oc n— >oo 

exist and do not depend on v. 

Though these "steady state" optimal errors cannot be calculated exactly, they are 
amenable to asymptotic approximations, as the one obtained by R.Khasminskii and 
O.Zeitouni in [7] and G.Golubev in [Sj. Suppose that the transition probabilities 
satisfy 

ye = f 1 i = J 

13 \eAtj-, i^j 

with a small parameter s S (0, 1), which controls the transitions rate of the corre- 
sponding slow chain X^ (note that the invariant measure of X s does not depend 
on e and equals /i). The observation process Y £ is given by with X replaced 
with X £ and ix £ n is the solution of (|1.3|) with Y and A replaced with Y £ and A e 
respectively. It is proved in [7j, that if all the Kullback-Leibler divergences 

@(9i II 9j) = / gi(u)log—(u)if)(du) 

JR 9j 

are finite and positive, the error probability 1 V s converges to zero as e — ► and 



^throughout superscripts are added to various quantities to emphasize their dependence on the 
corresponding parameter 
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Similar asymptotic holds for the minimal mean square error as shown in [SJ: 

£e = (e a* E g.) ( a * - °i) 2 j £ lQ g ^ ( x + °w) > e ^°- ( L5 ) 

These results give an idea of how fast the invariant measure M^{drj) concentrates 
around Ai®(drj) = ^2 i= ± HiO~ Pi (dr]), where are probability vectors with 1 at the 
i-th entry. 

In a sense the slow chain limit is the counterpart of the weak noise asymptotic 
a — > for the additive observation model (cf. ff 1 . 1 1) ) 

Y° = h(X n ) + a£ n , n > 1, (1.6) 

where £ is a sequence of i.i.d. random variables, independent of X, h is an § i— > R 
function and <r is the constant, controlling the noise intensity. Though less apparent 
in the discrete time setting, the analogy is complete for continuous time model, as 
explained in Section[2]below. In this paper the strong noise asymptotic is addressed, 
when the filtering probabilities 7r^ converge to the a priori distribution of the signal 
v n = (A*yV as <7 — > oo. Thus in the stationary case we deal with the concentration 
of M°(dr/) around M^idrj) = Snidrj) as a — > oo. The precise formulation of the 
results is given in Section [21 which are proved in Sections El and 0J 



2. Main results 

2.1. Discrete time. Let (X, Y a ) be the filtering model, with X being a finite state 
Markov chain on § with transition probabilities matrix A and initial distribution v 
and suppose that Y a is generated by ()1.6|) . 

Theorem 2.1. Assume that the probability law of £i has a bounded twice contin- 
uously differentiable density g(u) with respect to the Lebesgue measure on R with 
bounded continuous derivatives. Then the solution of (|1,3|) converges to v n = (A*) v 
as a — > oo and 

vUn - v n ) °' S '> Z n , n > 

a— >oo 

where Z n satisfies 

Z n = A*Z n _ 1 -(diagK)-y n z/*)/ J ^|4, Z = 0. (2.1) 

The following two theorems give asymptotic expressions for £ a and V° . 

Theorem 2.2. Assume that X is an ergodic chain and g satisfies the following 
conditions 

(ai) g(u) does not vanish on R, is bounded and has two bounded derivatives 
(a.2) there is a 5 > 0, so that 

f (~ — i — \ J 9(x)dx < oo, 

and 

f°° fmax\ v \ <s \g"(x + v)\\ 2 

/ / , x g{x)dx < oo. 
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Let I denote the Fisher information of g: 

/= /■« (ff'Qr)) 2 



< OO. 

Then the algebraic Lyapunov equation 

P = A* PA + (diag(/i) - fin*)hlh* (diag(/x) - /i//) (2.2) 

/ias a unique solution P in the class of nonnegative definite matrices with j Pij = 
and 

lim <7 2 (£°° -£ CT ) = a* Pa, (2.3) 

cr— >00 v 

where a is a vector with entries a\,...,a^ and £°° = n(a 2 ) — /i 2 (a) is i/ie a priori 
mean square error. 

Remark 2.3. The assumption ( tot and ergodicity of X guarantee uniqueness of 
the invariant measure M°{drj) (see [I]). The assumption ( toj is satisfied for many 
frequently encountered densities. For Gaussian density g(x) = (27r) -1 / 2 exp{— x 2 /2} 

|<?'(x)| \x\e~ x2 / 2 \x\ \x\ 



min|„|< 5 g(x + u) min| u |< 5 e~( x + u ) 2 / 2 mm\ u \< s e~ xu - u2 / 2 e -\x\s~5 2 /2 
and hence 

/ — n g(x)dx < / | ' ' e x /2 dx < oo 

J^ymm^sgix + u)] ~ \ e -\x\S-P/2j 

for any p > and not just p = 2 as required by the first part of (tot. Similarly 



+ max|,|< 5 |(x + V ) 2 -l|e-(^) 2 /2 w 2| ,, + , 2 



ff(cc + u) min^i^ e -( x ~ u ^ 2 / 2 



< (2x 2 + 25 2 + l)e 2 

and the second condition of (toj holds with any power p > as well. It is not hard 
to verify that ( toj ) also holds for e.g. Cauchy density g(x) = 7r _1 (l + x 2 ) -1 , which 
fails to have the first moment. 

Theorem 2.4. Assume that X is ergodic and £i is a standard Gaussian random 
variable, then for any continuous function F : M. d ^ R, growing not faster than 
polynomially, 

! F(a(r, - v))M°Mn) ^> EF(Z), (2.4) 

where Z is a zero mean Gaussian vector with covariance matrix P, defined by (12.21) 
with 1=1, In particular 

lim a(V°° - V a ) = EmaxZ,-, (2.5) 

where V 00 := 1 — max aiG gpj is the a priori error probability and J = {i : fa = 
maxj [ij } . If pi has a unique maximal atom, then for any integer p > 1 

lim a p (V°° -V a ) = 0. 

If the maximal atom of \i is not unique, then the right hand side of Q2.5|) is positive 
in general as the following example demonstrates. 
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Example 2.5. Let X be a binary chain with the transition matrix 

A=( x \ V), A, 7 €(0,1). 
The equation ()2.2jl is one dimensional and P := P\\ = P22 = —P12 = —P21 satisfies 

P = P(l - A - 7 ) 2 + l4l4(h! - h 2 f 

and hence 

Hl^-hjf (l-A) 2 (l-7) 2 (^i-M 2 
(A + 7 )(l- A+1-7) (A + 7 )(l- A + l-7) 5 
Now by Theorem l2~2l 

lim <r (£°° — £ a \ = (01 - a 2 ) 2 P. 

CT— >00 

By Theorem H3J if 7 / A 

lim ^(P 00 -P' 7 ) = 0, p>\ 

CT— >00 

and if 7 = A, 

lim a(V°° - V a ) = Emax(Z,-Z) = E\Z\ = 

CT-^OO V 

2^P r -^e~* 2 / 2 dx = l kl - h21 ■ 0.3839... 



V2n 4^(1 -A) 

2.2. Continuous time. The continuous time analogue of the aforementioned set- 
ting consists of a time homogeneous Markov chain with values in S, transition in- 
tensities A« and initial distribution v and the observation process Y a = {Y a )t^m. + 
satisfying 



Y t a = [ h(X s )ds + aB t , t > 0, 
Jo 



where h is an S ^ M function, a > is a real constant and B = (Bt)t>o is a Brownian 
motion, independent of X. We treat the continuous time case separately and hence 
use the same notations for transition intensities and transition probabilities, etc. 

The vector of conditional probabilities irt satisfies the Wonham filtering ltd sto- 
chastic differential equation (jUJ, see also [SJ) 

dir t = A*7r t dt + a- 2 (diag(^ t ) - Tr t ^)h(dY t - ir t (h)dt), (2.6) 

subject to 7To = v, where A is the transition intensities matrix, diag(x), x € M d 
stands for the scalar matrix with Xj on the diagonal, h is a column vector with 
entries h(ai) and x* is the transposed of x. 

Recall that X = (Xt)teR+ 1S ergodic, if exp(A) has positive entries or equivalently 
if all of its states communicate. For ergodic chains the Markov process irt has a 
unique invariant measure M°(dr]) for any a > (see |S1 II])- In the case d = 2 the 
exact expressions are known for both V and £ in terms of integrals with respect to 
the density of A4 n (dr]), which can be explicitly found by solving the corresponding 
Kolmogorov-Fokker-Plank equation (see jHj, [S]). In higher dimension the closed 
form solution for KFP equation is unavailable, which makes the direct analysis of 
(|2.6|) intractable. 

The slow chain X s is obtained by the time scaling Xf = X e t, t > and its 
transition intensities matrix equals eA. As in the discrete time the invariant measure 
of X £ is independent of e and solves A* \i = in S d ~ x . The asymptotic expressions 
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l|1.4|) and ()1.5() remain valid with @(gi \\ gj) replaced by (hi — hj) 2 /2 (see j7], j^l)- 
It is not hard to see, either by appropriate time change or directly from the KFP 
equation, that the weak noise asymptotic a — ► is obtained by replacing e = a 2 
in these expressions. The strong noise asymptotic also turns to be similar to the 
discrete time case: 

Theorem 2.6. The solution of Q2.6JI converges to ut = e ts * t v as a — > oo and for 

any p > 1 

a— >oo 

where Zt is the Gaussian diffusion process: 

dZ t = k*Z t dt + (diag(^) - v t vl) hdB t , Z = 0, (2.7) 

with B = o"~ 1 ^Y t <J — Jq 7rJ(/i)(is^ fremg i/ie innovation Brownian motion. If X is 

ergodic, the algebraic Lyapunov equation 

= A*P + PA + (diag(/i) - fifi*)hh* (diag(/z) - /i/x*) (2.8) 

ftas a unique solution P in the class of nonnegative definite matrices satisfying 
Y^ij Pij = and for any F : R d i— ► R, growing not faster than polynomially, 

! F(a(r, - n))M°{dri) ^> EF(Z), (2.9) 

where Z is a zero mean Gaussian random vector with covariance matrix P. 

Theorems 12.21 and 12.41 remain valid in continuous time case with obvious adjust- 
ments, namely 

lim a 2 {£°° - £ a ) = a* Pa and lim a(V°° - V a ) = ErnaxZ,-, 

(7— >00 V ' <7-*0O V ' j£j J 

where P is the solution of (|2.8|) and Z is the Gaussian vector defined in Theorem 
12.61 If the maximal atom of /i is unique, 

lim a p (V°° -V a ) =0, p>l. 

3. Proofs in discrete time 

3.1. Proof of Theorem 12.11 The entries of the diagonal matrix G(y) in (|1.3j) in 

case of the observations Q1.6|) have the form 

9 ( ~ o 7 " 1 ) ' * = 1 '-' d - 
To emphasize the dependence on cr write G a {y) and let 

T ff (y)a; = G CT (y)A*x/|G CT (y)A*x|, y G R, x G S* -1 . 
Since the density g(u) is continuous, for any z = 1, ...,d 

( YZ-hj \ ( f ^ h(X n )-hj \ P -g. s . 

and hence lim a ^ QO T cr (Y n )x = A*x, P-a.s. for any x G S^ 1 . Then for any fixed 
n > 1 

< = T°(YZ) o-.-o T°(Yf ) o v (A*)V = (3.1) 

cr — >oo 

Since both ir^ and f n are bounded, the convergence also holds in L p for any p > 1. 
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Let q° be the solution of 

q - = A*q-_ 1 -a- 1 (di ag (u n )-u n u*)h^-, ql = v. (3.2) 
The process A^ = cf^tt^ — q%) satisfies 

subject to A£ = 0. Denote ^\n-i = ^■* 7r n-i) tnen 

l^(^K|n-ll ^ 

ZU^n + ^h(X n ) - hj))^,^) 7fn|re - lW = 



where -ftT n are bounded random variables (recall that g"(u) is assumed bounded). 
Hence by (j3.1|) and continuity of g 



Qa ( Y n )K\n-l (j \ P-a.s. ,,. , , *\,S'(6i) 



I' 

Iterating (|3.3j) one gets lim^oo A^ = 0, P — a.s. for any fixed n > and the 
statement of the theorem follows: 

<r« - f n ) = cr« - g£) + a(q° - u n ) P ~"' s '> Z n , 
where Z n := cr(q^ — v n ) clearly satisfies (|'2.1() . which doesn't depend on a. □ 

3.2. Proof of Theorem 12.21 Note that to verify (|2.3|) one should first take the 
limit n — > oo and then a — > oo and thus cannot use the statement of Theorem 12. II 
per se. The proof relies on stability property of the matrix A, provided by ergodicity 
of X. 

As shown in [3] the Markov process (X,ir a ) has the unique invariant measure 
A4° '(dx,drj) if X is ergodic and assumption ( jgjt is satisfied. In particular Ai^(dr]) = 
Yli=i -M. a ({cii} , dr/) . If the equation (|1.3|) and X is started from a random variable 
with distribution Ai a (dx,dr)), the process -k" = (j^n)n>o is stationary, which is 
assumed hereafter. 

As in (|3.1|) lim (7 _ >00 = (A*) n 7TQ , P — a.s. and so for any e > and any m > 
lim P(K - Ml > e) = lim P(|< - fi\ > e) < 

CT— *00 (T-^OO 

lim PAtt" - (A*) m <| > e/2) + lhn P(|(A*) m < - d > e/2) (3.4) 

CT— »00 (T^OO 

where the latter convergence holds since (A*) n x — > /x for all x £ by ergodicity 
of A". 
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Let q^ denote the solution of (cf. (|3.2)) ) 

& = A*g£-i - ( 7- 1 (diag( M ) - M/^^M = 

We use the notations, introduced in the previous section, to denote random pro- 
cesses, playing the same role as in the proof of Theorem 12.11 but defined differently 
to fit the stationary setup under consideration. 
The process A^ = <r(vr^ — q°) satisfies (cf. (|3.3)) 1 

(diag(/x) - /V>4rr := A*A£_i + 0* 

subject to Aq = o"(7Tq — fj,). Note that since the Fisher information is finite and 7r£ 
is stationary, for any fixed a > 0, E0£0£* = E9%9%* := T a and hence Ql := EA£A£* 
satisfies 

Q£ = A*Q£_ 1 A + P J , n>l, (3.5) 
subject to Qq = c 2 E(7Tq — /i)(vro — fJ,)*- If -X" is ergodic, A* is a stability matrix, 
when restricted to the subspace {x G M d : Ej=i x i = *-*}• Since A^ belongs to this 
subspace for all n > 0, the Lyapunov equation (|3.5jl has a bounded solution, which 
converges to the unique limit Q a = J2m=o A*T°'A TO . 
For brevity set -kZ q = A*7Tq and define 



" I^WKiol ^ 

Then 

a U) = 0" i ; ; H ^unW = 

g{i l + a-\h(x l )-h i ))-Y J Ua{^ + ^ 1 (KXi)-h ] ))^{j) 

a — : : : n ll0 {i) 



£^s(£i + ^-Wi)-MK|o(i) 
V(fi)(^-E?=i^<| (i)) 



<io(0+ 



£^i^i + ^Wi)-**)K|ok') 110 
2(7 Ei=i 9(6 + Wi) - MK|oO') 

where the latter holds by the mean value theorem with |<x,-| < — and 

0j G [0, 1]. Since 5" is bounded and by (j3.4|) 7r^ — ► /x in probability as a — > 00 

*•«>-=- 9 ' <6) ' 



Note that for a > maxjj \hi — hj\/5, 



\g'(Zi)\ < W&)\ 



E?=i <?(6 + ^(M^i) " hj))< V) min| M |< 5 5 (ei + u) ' 
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where by the assumption ( jg^t the right hand side is square integrable. Analogously 
for sufficiently small a, 

\g"(£i + ai/a)\ max^i^ \g"(£i + v)\ 



E?=i3(£i + ^HM*i)-^)K|o(i) " ^\u\< s g{ii + u) 1 

with a square integrable right hand side. Hence by the Lebesgue dominated conver- 
gence 1)3.6(1 implies 

L 2 ^ *N,5'(ll) 



(diag(/i) - 



5(6) 
and in turn 

lim tr(T CT ) = =► lim Q u = 0. (3.7) 

O" — »00 f7— >-00 

On the other hand, the sequence Z n = — /i) does not depend on a and satisfies 

Z n = A*Z n _i-(diag(^)-^*)fe^^, Z = 0. (3.8) 
Again by the stability property of A* on the subspace {x E R d : X^f =i x j = 0} 



lim EZ n Z* = P, 

n— >oo 



where P uniquely solves (|2.8jl in the class of nonnegative matrices with £^ ■ Pij = 0, 
which is a well known property of the Lyapunov equation for stable matrices (see 
e.g. pQ). Hence, 

a 2 E« - M )« -fi)*-P = <r 2 E« - M )« - M )* - lim Z n Z; = 

n— >oo 

lim EA£A£* + lim E(A£Z* + Z* n ^ n ) ^> (3.9) 

n— >oo n^oo 

where the latter convergence holds by 1)3. 7[) . This in turn implies ()2.3() : 

3.3. Proof of Theorem 12.41 For standard Gaussian £i> p'{ x )/p( x ) = ~~ x an d 
7=1, hence the process Z n , defined in (|3.8|) is Gaussian. By the Remark 12 .31 Gauss- 
ian density satisfies the assumption ( jg^t of Theorem 12.21 with square integrability 
replaced by integrability to any power p > 1 and hence, similarly to (|3.9|) . for any 
continuous F : R d i— > R with the norm bounded by a polynomial function of any 
finite order 

EF(<7«- M )) ^>BF(Z), 

where Z is a zero mean Gaussian vector with covariance matrix P, defined by l|2.2|) . 
Let J = {i : fj-i = maxj and assume J for definiteness. Then 

a(V°° — V a ) = 0"(Emax7TQ (i) — max/ij) = E<r max(7TQ (i) — //i) = 

Emax (<t(ttq (i) — pLi) + a{fJLi — /ii)) owoo ) EmaxZj, 

where the convergence holds by Q2.4[) . since maxj(xj), x S R rf is a continuous function 
and //j — Hi < for i £ J . 
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Suppose now that [i\ is the unique maximal atom of \i and let r = min^i \fj,\ — 
fj,j\ > 0. Let A a := {\ttq — < r/2}, 1a„ be the indicator function of A a and 
A% = n\A a . Then 

max<(z) = + max<(i) = tt£(1) + \ A% (max<(z) - <(1)), 

a;GS a;Gs diGs 

Hence for any two integers q> p> 1, 
cr^lEmaxTr^i) — max/iJ = 

aP\ (E<(1) - m ) + El A c ( max<(i) - tt ct (1)) | = 

o*E1ac | max<(z) - <(1)| < 2^ E| ^ ~ ^ ^> 0, 
since by (|2.4[) . the limit linio-^oo o" 9 E|7Tq — exists and is finite. □ 

4. Proofs in continuous time 



The continuous time filter l|2,6j) has a convenient innovation structure, which 
simplifies the proofs, which are only sketched below. 

4.1. Proof of Theorem 12.61 Since u t = exp(A*i)i/ solves i>t = A*u t , vq = u, the 
process 5" := ir£ — v% satisfies 

dS? = A*5°dt + a' 1 (diag«) - «*) hdB t , 5% = 0, 

and hence 

<5f = (J" 1 / e A * (i - s) (diag«) -«*)MB S . 
Since the integrand is continuous and bounded for any t > 0, 



lim (5f = 0, P - a.s. 

a— »oo 



The convergence holds in L p , p > 1 as well, since the integrand of the stochastic 
integral is uniformly bounded in a and hence |<5f | is uniformly integrable to any 
power as a — > oo. Let be solution of the linear SDE 

dq? = A*qfdt + a" 1 (diag(^) - v t vf) hdB t , q% = v. 

The process = o(jt1 — q") satisfies 

dA a t = A*A°dt + (r«) - T(u t ))hdB t: Aq = 0, (4.1) 

where T(x) = diag(x) — xx* is defined for brevity. Then 

A a t = I e A *( i - s )(r«) -Y(y 8 ))hdB a 0, P - a.s and in L p , 

since T(-) is continuous, 7rf and are bounded and ivf — > vt P-a.s. as a — > oo. The 
process Z t = a{q1 — v t ) satisfies (|2.7[) and thus 

a« - ut) = a« - gf ) + a(<# - ^) Z t . 

If X is ergodic, A* is a stability matrix on {x E 1R : £^ Xj = 0}, which is an invariant 
subspace of (|4.1|) and (|2.7|) . Then by the very same arguments, used in the proof of 
Theorem 12.21 and taking into account the integrability properties of the stochastic 
integral with respect to B, one verifies (|2.9|) . □ 
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